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0 Pipeline system with real-time mulriresolutlon data processing. 



0 In a data processing pipeline system comprising: 

- a cascade of data processing modules for the 
processing of successive groups of data units which 
are relevant to the processing and which are pre- 
sented to the system input, said groups being sepa- 
rated from one another by time slots. 

- control means for supplying the modules with at 
least one control signal for controlling the processing 
by the modules, which control signal occurs in syn- 
chronism with the presence of each group on the 
system input. 

a loss of information occurs due to processing de- 
lays if data transport between the modules tal<es 
place in a time slot because of the absence of 
control signals. Therefore, the pipeline system com- 
prises: 

- monitoring means for stopping the processing by 
the modules in a time slot on the system input in the 
absence of the control signal. When the control 
signal (for example, sampling signals) are of a peri- 
odic nature within each processing cycle, indicated 
by a data valid signal, a length (expressed in data 
units) of each cycle equals an Integer multiple of a 



length of a period of each control signal. In that case 
the control signals do not comprise a phase jump 
between successive cycles, so that further loss of 
information is avoided. 
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PIPELINE SYSTEM WITH REAL-TIME MULTI-RESOLUTION DATA PROCESSING. 



The invention relates to a data processing 
pipeline system comprising: 

- a cascade of data processing modules for the 
processing of successive groups of data units 
which are relevant to the processing and which are 5 
presented to a system input, said groups being 
separated from one another by time slots; 

- control means for supplying the modules with at 
least one control signal for controlling the process- 
ing by the modules, which control signal occurs in w 
synchronism with the presence of each group on 

the system input; the invention also relates to a 
method of processing in a pipeline structure suc- 
cessive groups of data units, separated from one 
another by time slots, under the control of at least ;5 
one control signal which appears in synchronism 
with the groups of data units prior to a processing 
operation in the pipeline structure. A pipeline sys- 
tem of this kind is known from European Patent 
Application EP-A 0285192 (PHN 12.057). The 20 
known pipeline system is an image processing 
system in which an image is generated by means 
of the frame scan method. Therein, an image is 
composed of successive lines, each of which com- 
prises successive pixels. The time slots are pro- 25 
duced, for example by the line flyback and frame 
fly-back occurring during image acquisition (see, 
for example the Figs. 2 and 1 of EP-A-02851 92). In 
the known system the control signals are applied to 
the modules in parallel and are delayed in each 30 
module by a relevant delay generator in accor- 
dance with the cumulative delay incurred by the 
already processed data due to the processing in 
the preceding modules. The delay of the control 
signals maintains the synchronization between the 35 
data on the one hand and the control signals on the 
other hand. 

Even though the known system can be readily 
reconfigured and has a simple set-up, the solution 
chosen to solve the synchronization problem im- 4o 
plies that each module must be provided with a 
delay operator for the control signals in order to 
prevent, for example, the occurrence of time slots 
from disturbing the data processing. 
Another solution for the mutual synchronization of 45 
data and control signals could consist in delaying 
the data per module so that they remain synchro- 
nized with the control signals. Like in the previous 
solution, additional operators are then required for 
delaying the data. An additional drawback consists so 
in that the overall pipeline processing time is thus 
increased. 

Another solution, for example in the case where the 
control signals are sampling signals, would be the 
resampling of the data. This has the drawback that 



resampling requires additional arithmetic units for 
interpolation, thus introducing additional processing 
delays in the system. 

Therefore, it is inter alia an object of the inven- 
tion to provide a pipeline^system of the kind set 
forth which enables real-time data processing with- 
out necessitating the use of additional components 
such as delay or reconfiguration operators. To 
achieve this, a pipeline system in accordance with 
the invention is characterized in that it furthermore 
comprises: 

- monitoring means for stopping the processing by 
the modules in a time slot on the system input in 
the absence of the control signal. 
By deactivating the modules for the duration of a 
time slot, no data transport takes place between 
the modules during an interval in which the mod- 
ules do not receive control signals, for example 
sampling signals. The presence per se of time 
slots between successive groups of data units in 
that case does not lead to loss of information 
because the time slots do not enter the cascade so 
to speak. 

It is to be noted that the sole requirement imposed 
as regards the control signals consists in that these 
signals must appear In synchronism with the pres- 
ence of the groups of data units before the data 
units are entered into the cascade. One method of 
satisfying this requirement, for example consists in 
the coupling of a single control signal source to the 
modules via a characteristic filter for each module. 
The filtersTransform the source signal into control 
signals which are appropriate for each module and 
which are mutually synchronized because they 
have been derived from the same source signal. 
This method offers the advantage that the pipeline 
system requires only one source for the control 
signal, the mutual synchronization of the control 
signals being automatically achieved. When a con- 
trol signal, for example a sampling signal, is of a 
periodic nature within a processing cycle which is 
indicated by way of a data valid signal, loss of 
information is liable to occur if the processing of a 
block of coherent data units exceeds the boundary 
between two successive cycles. This possible loss 
is caused by a phase jump between the control 
signal at the end of a cycle and the control signal 
at a beginning of the next cycle. Should such 
phase jumps occur, the result of a processing 
operation on a block of data units, within one cycle 
under the control of the rejevant control signal, will 
differ from the result of the same operation on a 
further, identical block of data units when the pro- 
cessing operation exceeds the boundary between 
successive cycles. 
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Therefore, it is another object of the invention 
to provide a pipeline system in which the occur- 
rence of further loss of information due to the 
occurrence of said phase jumps is avoided. To 
achieve this, a pipeline system in accordance with 
the invention is further characterized in that a first 
control signal is a data valid signal which is indica- 
tive of a window of data units relevant to the 
processing operation, which window has a dimen- 
sion equal to a first number of successive data 
units, and in that at least one further control signal 
is periodic with a period equal to a further number 
of successive data units, the first number being an 
integer multiple of the further number. 
This implies that a phase of the further control 
signal at an end of each processing cycle equals a 
phase at a beginning of the subsequent cycle. 
When the control means supply different modules 
with control signals having periods of mutually dif- 
ferent lengths, the length of a processing cycle, 
indicated by the data valid signal, should be equal 
to a common multiple of the lengths of the relevant 
periods in order to avoid loss of information due to 
the occurrence of phase jumps. Preferably, the 
dimension of the window indicating the data rel- 
evant to the processing operation is adjustable. 
Depending on the desired periods of the control 
signals, one or more dimensions of the window can 
be adjusted to the common multiple. 

The highest sampling frequency is that at 
which the data units are supplied. This highest 
sampling frequency is assumed to be equal to i 
for the sake of convenience. Known systems use 
the conventional sampling frequencies, for example 
1/2, 1/4. 1/8. because therein a cycle has a length 
amounting to a number of data units equal to a 
power of 2. In that case one of each time 2. 4 or 8 
data units is selected for processing. In a pipeline 
system in accordance with the Invention in which, 
the phase continuity is ensured in the described 
manner, sampling frequencies such as, for example 
2/5 and 3/7 are also possible. In that case 2 and 3 
data units are selected from 5 and 7 data units, 
respectively, for processing. Extension to sampling 
frequencies which are not powers of 1/2 or mul- 
tiples thereof increases the range of application of 
the pipeline system. Successive sampling oper- 
ations executed at different sampling frequencies 
(resolutions) can be realised on a real-time basis m 
a pipeline system in accordance with the invention. 
It is to be noted that the lengths of successive 
processing cycles need not necessarily be iden- 
tical. . ^ 
The invention will be described in detail 

hereinafter with reference to a drawing; therein 
Fig. 1 shows an embodiment of a pipeline sys- 
tem in accordance with the invention. 
Fig. 2 illustrates the processing delay. 



Fig, 3 shows an embodiment in which loss of 
information occurs due to the phase jump in the 
control signal at a transition between successive 
cycles, and 

5 Fig. 4 shows an embodiment in which loss of 
information at the transition between successive 

cycles is avoided. 
Fig. 1 shows an embodiment of a pipeline 
system in accordance with the invention in the form 
70 of an image processing system 10 . The input unit 
of the system is formed by a camera unit 12 which 
outputs image information in successive image 
lines with successive pixels to a cascade of image 
processing modules, three modules 14. 16 and 18 
75 being shown. The modules are connected vta a 
bus 20 for data transport. The image information 
processed by the module 14 is transported to the 
module 16 via the bus 20. Similarly, the image 
information processed by the module 16 is trans- 
20 ported to the module 18 via the bus 20. Each 
module executes a predetermined operation on the 
image data, which operation is repeated per image 
or per line. Using a sampling signal, the data 
appropriate for the relevant operation is selected 
25 from the stream of pixels. To this end there is 
provided a control line 22 on which clock pulses 
appear in synchronism with the delivery of pixels 
by the camera unit 12. The frequency of these 
clock pulses is the highest frequency occurring in 
30 the system. If the frequency of the sampling signal 
applied to a relevant module equals the frequency 
of these clock pulses, the relevant operation has 
the highest resolution. Execution of operations with 
a lower resolution is possible by deriving sampling 
35 signals of lower frequency from the clock pulses on 
the control line 22. To this end there are provided 
sampling circuits 24. 26 and 28 which are arranged 
between the control line 22 on the one side and the 
modules 14, 16 and 18 on the other side and which 
40 receive the clock pulses in parallel. The sampling 
circuits 24. 26 and 28 comprise, for example fre- 
quency dividers or clock pulse modulo counters 
having a predetermined counting range. A clock 
pulse modulo counter then supplies sampling 
45 pulses to the relevant module when it reaches a 
predetermined count. 

A module receives, processes and outputs data in 
the form of pixels. Processing requires some time: 
the processing time. Generally low-resolution pro- 

50 cessing operations require more time than high- 
resolution processing operations. 

Because of the processing time, the image 
output by the relevant module has incurred a delay 
with respect to the image received by the relevant 

55 module. This is illustrated on the basis of an exam- 
ple in Fig. 2. An image 40 received by a module 42 
is processed and output as an image 44; during 
this operation it incurs a delay with respect to the 
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image 40. In Fig. 2 the delay equals one image line 
46 plus two pixels 48 and 49. 
For the sake of clarity it Is assumed that the image 
40 is the Input image of the module 14 In Rg. 1 
and that the image 44 is the processed input 
Image, successive image lines of the image 40 are 
separated by a line fly-back time, successive im- 
ages being separated by a frame fly-back time of 
the camera unit 12. These fly-back times constitute 
time slots which would propagate through the sys- 
tem together with the processed images. The clock 
pulses on the control line 22 are synchronized with 
the stream of pixels on the input of the module 14. 
These clock pulses are also group-wise separated 
by time slots. When information relevant to the 
processing operations of the output image 44, such 
as the indicated shaded area, is applied to a next 
module 16 during the occurrence of a time slot, at 
least a part of said Information will not be accom- 
panied by sampling signals derived from the clock 
pulses, because the sampling signals are applied 
in parallel to the modules in synchronism with the 
clock pulse pattern. In order to avoid loss of in- 
formation, the processing is stopped during the 
occurrence of a time slot. To this end. the system 
10 comprises monitoring means 30 which stop the 
processing by all modules 14, 16 and 18 during the 
occurrence of a time slot. The modules receive the 
monitoring signals via the monitoring lead 32 for 
this purpose. Becauie"the modules are deactivated 
during the occurrence of a time slot, the time slot 
win not be noticed by the processing. 
The modules 14. 16 and 18 receive sampling sig- 
nals which recur in line cycles and frame cycles, 
related to the stream of pixels on the input of the 
module 14. In the present embodiment the number 
of pixels, and hence the number of clock pulses on 
the control line 22. is the same for each tine. It 
follows therefrom that the pattern of sampling sig- 
nals, derived from the clock pulses by the sam- 
pling circuits 24, 26 and 28, is also recurrent in 
cycles of the same length. 

Fig. 3 shows an example of the occurrence of 
loss of information if the processing of pixels is 
controlled by means of sampling signals originating 
from two successive cycles. The Figure shows 
successive cycles 50 and 52 of clock pulses CK as 
they occur on the control lead 22, and of sampling 
signal SA derived from the clock pulses. The sam- 
pling signal SA has a period of five clock pulses 
(pixels) and comprises two sampling pulses in each 
period. For the sake of clarity the successive 
periods of the sampling signal SA are denoted by 
the reference letter P and successive, staggered 
bars. The cycles 50 and 52 are separated by a 
time slot SL in which the processing by the mod- 
ules is stopped. When the processing is resumed, 
the. beginning of a cycle (52) is identified by the 



end of the preceding cycle (50). 

Fig. 3 also shows an object 60 having a length 
of ten pixels. When this object is sampled with the 
sampling signal SA. the pixels denoted by a cross 

5 (four pixels) are selected and processed. For as 
long as processing takes place within a cycle, four 
pixels are selected by the sampling signal SA. 
regardless of the phase of the "signal SA at the 
beginning of the object 60. Also shown is an object 

w 62 which has the same length and whose process- 
ing is interrupted by the time slot SL An apparent 
widening of the object then occurs, tjpon sampling 
by the signal SA. five instead of four pixels are 
selected. It appears as if the object is longer than 

15 ten clock pulses. The virtual widening of the object 
is caused by the phase jump in the sampling signal 
SA at the transition to a next cycle. A kind of 
syncope occurs in the regularity of the sampling 
signal SA. 

20 In Fig. 4, in which attributes corresponding to 

Fig. 3 are denoted by corresponding references, it 
Is ensured that a processing cycle comprises an 
integer number of periods of the sampling signal 
SA, so that the phase of the signal SA at the end of 

25 a cycle (50) is the same as the phase at a begin- 
ning of a next cycle (52), This avoids the virtual 
widening or narrowing of an object due to a transi- 
tion between successive cycles. When mutually 
different periodic sampling signals are applied to 

30 the modules 14, 16 and 18, the described loss of 
information will not occur, subject to the condition 
that the cycle length amounts to an integer and 
common multiple of the length of the periods of 
these three sampling signals. 

35 it is to be noted that a method as described 

above can be readily implemented by means of 
programmed computer means. 

40 Claims 

1 . A data processing pipeline system comprising: 

- a cascade of data processing modules for the 
processing of successive groups of data units 

45 which are relevant to the processing and which are 
presented to a system input, said groups being 
separated from one another by time slots, 

- control means for supplying the modules with at 
least one control signal for controlling the process- 
so ing by the modules, which control signal occurs in 

synchronism with the presence of each group on 
the system input. 

characterized in that the pipeline system also com- 
prises: 

55 - monitoring means for stopping the processing by 
the modules in a time slot on the system input in 
the absence of the control signal. 

2. A data processing pipeline system as claimed in 
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Claim 1. characterized in that a first control signal 
is a data valid signal which is indicative of a win- 
dow of data units relevant to the processing opera- 
tion, which window has a dimension equal to a first 
number of successive data units, and in that at s 
least one further control signal is periodic with a 
period equal to a further number of successive 
data units, the first number being an integer mul- 
tiple of the further number. 

3. A data processing system as claimed in Claim 2, w 
characterized in that the first number is an odd 
multiple of the further number. 

4. A data processing system as claimed in Claim 2 
or 3, characterized in that at least one dimension of 

the window is adjustable. '5 

5. A data processing system as claimed in Claim 2, 
characterized in that the control means comprise a 
control signal source for supplying a source signal, 
couplings between the control signal source and 

the modules, and a filter In at least one coupling in 20 
order to derive a control signal from the source 
signal. 

6. A data processing pipeline system as claimed in 
Claim 1. 2. 3. 4 or 5. constructed as an image 
processing system suitable for the processing of 25 
successive groups of data units obtained by the 
sampling of an image. 

7- A method of processing successive groups of 
data units, separated by time slots, in a pipeline 
structure under the control of at least one control . 30 
signal which occurs in synchronism with the groups 
of data units prior to a processing operation in the 
pipeline structure, characterized in that the pro- 
cessing is stopped, in the absence of the control 
signal, during the occurrence of the time slots 35 
preceding the processing. 

8. A method as claimed in Claim 7. characterized 
in that a first control signal is a data valid signal 
which is indicative of a window of data units rel- 
evant to the processing, which window has a di- 40 
mension equal to a first number of data units, and 
in that at least one further control signal is periodic 
with a period equal to a relevant further number of 
data units, the first number being an odd multiple 
of the relevant further number. 
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